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ABSTRACT 


Methods  that  can  be  used  to  form  a  priority  ranking 
of  projects  having  multiple  attributes  are  illustrated. 
Advantages,  disadvantages,  and  pertinent  characteristics  of 


each  of  the  methods  are  discussed. 
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PRIORITY  RANKING  OF  PROJECTS  HAVING  MULTIPLE  ATTRIBUTES 


INTRODUCTION 


In  reponse  to  a  request  from  the  Director  of  Land 
Requirements  ( DLR ) ,  a  literature  search  was  undertaken  for 
existing  methods  that  could  be  used  to  construct  a  priority 
ranking  of  projects  having  multiple  attributes.  This  note 
discusses  the  use  of  several  such  methods  which  may  be 
relevant  to  DLR's  request,  and  could  be  considered  worthy 
of  use  on  a  trial  basis.  Readers  interested  in  detailed 
specifics  of  any  of  the  methods  should  refer  to  the  original 
papers . 

THE  PROBLEM 


A  set  of  objects  (or  alternatives,  or  projects)  are 
to  be  ranked,  taking  into  account  their  desirability  with 
respect  to  a  number  of  different  attributes.  The  attributes 
do  not  necessarily  have  equal  importance.  For  instance,  the 
objects  to  be  ranked  could  be  possible  acquisitions  such  as 
new  vehicles,  additional  tanks,  improved  ammunition  or  a 
command  and  control  system.  Attributes  to  be  considered  in 
forming  a  ranking  could  be  characteristics  such  as 
acquisition  expense,  operating  expense,  safety  of  personnel 
operating  the  equipment,  effectiveness  of  the  equipment  in 
its  role,  and  requirement  for  the  equipment. 
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In  the  case  of  the  Directorate  of  Land  Requirements, 
they  have  a  number  of  projects  (new  acquisitions,  upgrades 
of  existing  equipment,  more  items  of  existing  equipment, 
etc. )  for  which  they  would  like  to  get  approval,  and 
allocation  of  funds.  Whether  or  not  funds  are  allocated  to 
a  project  depends  on  how  well  it  is  justified.  However,  DLR 
has  limited  manpower  with  which  to  staff  project  proposals. 

A  major  project  may  require  several  man-years,  or  more,  of 
highly  qualified  technical  personnel  in  order  to  be 
adequately  justified.  Not  all  of  the  projects  can  be 
properly  staffed,  and  a  ranking  of  the  projects  (indicating 
overall  order  of  preference)  would  provide  guidance  with 
regard  to  the  allocation  of  manpower  to  them. 

Several  methods  that  could  be  considered  for  the 
construction  of  priority  rankings  are  presented.  The 
methods  are  based  on  Multiple  Objective  Optimization  Theory, 
Relative  Position  Estimate,  fuzzy  set  theory,  and  conjoint 
scaling . 

AN  APPROACH  BASED  ON  MULTIPLE  OBJECTIVE  OPTIMIZATION  THEORY 

This  approach  is  taken  from  Reference  1.  In  the 
example  which  is  worked  out  below,  the  approach  alternates 
between  two  processes:  first,  an  examination  of 

alternatives  to  see  which  ones  clearly  dominate  others,  and 


T 
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second  a  "trade-off"  of  some  attributes  according  to 
weighting  factors  chosen  by  the  decision  maker.  (These 
processes  will  be  described  in  detail  shortly.)  As  the 
decision  maker  alternately  applies  these  processes,  one 
after  the  other,  a  priority  ranking  emerges  in  finer  and 
finer  detail.  Such  a  ranking  of  alternatives  (which,  for 
the  sponsor,  would  correspond  to  projects  or  acquisitions) 
can  be  used  to  assist  in  deciding  which  ones  are  most 
desirable . 

Consider  a  set  of  8  possible  equipment  acquisitions, 
each  one  of  which  is  rated  according  to  5  attributes  ('a', 
'b',  'c',  'd'  and  '  e')  on  a  scale  of  0  to  10  as  represented 

in  Table  I.  In  each  case,  an  attribute  rating  of  10  is  most 
desirable,  and  a  rating  of  0  least  desirable.  So,  for 
attribute  'a',  a  rating  of  10  indicates  a  most  desirable 
(low)  acquisition  cost,  while  for  attribute  '  c'  it  would 
indicate  a  highest  safety  level.  The  attributes  are  assumed 
to  be  independent  of  one  another.  So,  a  high  or  low  rating 
on  one  attribute  for  a  given  project  should  not  automatically 
imply  the  necessity  of  a  high  or  low  rating  for  another 
attribute  of  the  same  project. 

All  8  types  of  equipment  are  considered  desirable, 
but  it  is  not  possible  to  acquire  all  of  them.  So,  a 
priority  ranking  (overall  most  desirable  equipment  down  to 
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TABLE  I 


a 

b 

c 

d 

e 

Alternative  1 

9 

10 

10 

8 

7 

2 

3 

9 

4 

9 

8 

3 

3 

8 

0 

2 

5 

4  i 

8 

7 

8 

4 

0 

5 

5 

1 

3 

10 

2 

6 

5 

4 

2 

9 

1 

7 

0 

8 

3 

5 

5 

8 

2 

0 

2 

9 

8 

a:  acquisition  expense  rating 

b:  operating  expense  rating 

c:  safety  rating 

d:  effectiveness  rating 

e:  requirement  rating 


Figure  1 
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overall  least  desirable  equipment)  would  help  in  deciding 
which  ones  are  most  worthy  to  obtain.  The  difficulty  is,  of 
course,  that  such  a  ranking  cannot  immediately  be 
constructed  from  the  information  in  Table  I. 

The  first  step  in  the  Multiple  Objective  Optimization 
Theory  (MOOT)  based  approach  is  to  examine  the  eight 
alternatives  for  any  clearly  dominating  relationships.  For 
instance,  alternative  2  dominates  alternative  3  since  each 
of  its  five  attribute  values  is,  respectively,  bigger  than 
or  equal  to  alternative  3's  attribute  values.  With  regard 
to  alternative  1  versus  alternative  2,  neither  of  them 
dominates  the  other  since  alternative  1  has  the  higher 
values  for  attributes  'a',  'b',  and  'c'  while  alternative  2 

has  the  higher  values  for  attributes  'd'  and  'e 1 .  The 
domination  pattern  corresponding  to  Table  I  is  presented  in 
Figure  1.  A  path  downwards  from  one  alternative  to  another 
indicates  a  clear  domination  of  the  upper  alternative  over 
the  lower  one.  This  pattern  gives  a  small  amount  of 
information  with  regard  to  a  priority  ranking.  For 
instance,  the  highest  ranking  alternative  must  be  one  of  1, 
2,  5  or  6  since  all  the  rest  are  dominated  by  alternative  1 
and/or  2.  Alternative  1  is  clearly  preferable  to 
alternatives  3,  4,  and  7,  while  alternative  2  dominates  3, 


7 ,  and  8 
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The  next  step  is  to  choose  suitable  attributes  (or 
groups  of  attributes)  to  "trade-off"  weights  between  them. 
Ideal  attributes  to  be  considered  would  be  ones  that  are  a) 
the  least  critical,  or  b)  components  of  some  overall  aspect, 
so  that  the  choosing  of  relative  weights  is  easier.  By 
doing  the  easier  or  less  critical  trade-offs  first,  the 
problem  structure  is  simplified  by  the  time  the  more 
difficult  trade-offs  must  be  made. 

In  the  example,  attributes  'd'  and  'e'  are  chosen  to 
be  combined.  (Effectiveness  and  requirement  attributes 
together  can  be  considered  to  be  the  aspect  of  operational 
importance) .  The  decision  maker  gives  attribute  'd'  a 
weight  factor  of  .4,  and  attribute  'e'  a  weight  factor 
of  .6.  On  alternative  1,  for  example,  the  combining  of  'd' 
and  'e'  results  in  the  value  (.4) (8)  +  (.6) (7)  or  7.4. 

Table  II  gives  the  ratings  of  the  8  alternatives  after 
performing  the  trade-off.  The  steps  of  1)  analyzing  the 
domination  relationships  and  2)  performing  trade-offs  can  be 
repeated  until  a  final  answer  emerges.  One  of  the  big 
advantages  of  the  MOOT  method  is  that  the  implications  of 
trade-offs  can  be  seen  as  the  method  progresses,  so  the 
decision  maker  has  an  awareness  of  the  effect  of  his  choices 
for  relative  weights.  Figure  2  shows  the  domination 
relationships  derived  from  Table  II .  All  of  the  dominating 
relationships  from  Figure  1  still  hold  (as  they  must)  but 
new  dominations  are  introduced  by  the  combining  of 
attributes  'd'  and  'e'.  It  is  now  clear,  for  instance  that 
the  most  preferred  alternative  is  either  1  or  2 . 


I 


P34944.PDF  [Page:  12  of  48] 


-  8  - 


The  decision  maker  next  combines  attribute  'c'  (with 
a  weight  of  .5)  with  the  'd-e 1  attribute  (again  with  a 
weight  of  .5)  from  Table  II  to  form  Table  III.  Figure  3 
shows  the  resulting  domination  relationships.  From  Figure  3 
it  is  seen  that  alternative  1  is ,  as  a  result  of  the 
trade-offs  that  were  made,  ranked  first.  Both  alternatives 
2  and  4  are  candidates  for  second.  If  the  decision  maker 
felt  he  could  do  so,  he  might  want  to  choose  elements  from 
Figure  3  one  by  one,  using  his  judgement  in  relation  to  the 
information  in  Table  III,  in  order  to  construct  a  final 
ranking.  Alternative  1  is  ranked  first.  Say,  using  Table 
III  he  decides  alternative  2  is  ranked  second.  Then, 
alternatives  3,  4,  7  and  8  would  have  to  be  compared  in 
order  to  choose  one  of  them  as  third  ranked. 

If  the  decision  maker  is  not  prepared  to  compare 
alternatives  directly  from  Table  III,  then  he  will  have  to 
consider  another  trade-off.  Since  the  final  ranking  may  be 
very  sensitive  to  his  choice  of  trade-off  weights,  it  is 
reasonable  for  him  to  try  out  different  ranges  and 
combinations  of  weights,  to  see  how  greatly  changes  in 
weights  affect  the  final  ranking.  If  the  answer  does  not 
change  readily  in  response  to  moderate  weight  alterations, 
then  the  decision  maker  may  more  readily  put  faith  in  the 
results.  On  the  other  hand,  if  small  weight  changes  were  to 
make  drastic  differences  in  the  final  ranking,  then  the 
decision  maker  should  consider  carefully  possible 
ramifications  of  the  weights  he  chooses. 


T 
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TABLE  III 


a:  acquisition  expense  rating 

b:  operating  expense  rating 

c-d-e:  combined  safety,  effectiveness,  and  requirement  rating 


I 


Figure  3 


P34944.PDF  [Page:  15  of  48] 


-  11  - 

The  decision  maker  specifies  a  weight  of  .3  for 
attribute  'a',  .3  for  attribute  'b',  and  .4  for  the  combined 
'c-d-e'  attributes.  The  trade-off  results  are  shown  in 
Table  IV  and  the  final  overall  ranking  is  shown  in  the 
domination  relationships  of  Figure  4. 

In  order  to  get  a  feeling  for  how  sensitive  the 
ranking  is  to  these  weights,  two  comparison  trade-offs  are 
generated.  The  first  has  weights  of  .2  for  'a',  .2  for  'b', 

and  .6  for  the  combined  'c-d-e'  attributes.  The  results  are 
shown  in  Table  V  and  Figure  5.  The  second  has  weights  of 
.33  for  'a',  .33  for  'b',  and  .34  for  'c-d-e'.  The  results 
are  shown  in  Table  VI  and  Figure  6. 

Figures  4  and  6  are  almost  identical,  so  the 
different  weightings  for  Tables  IV  and  VI  have  little 
bearing  on  the  final  ranking. 

Figures  4  and  5,  on  the  other  hand,  do  have  some 
considerable  differences.  The  first  three  alternatives  in 
each  ranking  are  the  same,  but  not  in  an  identical 
ordering.  If  the  decision  maker  were  interested  in  the 
first  five  alternatives,  then  the  difference  between  the 
rankings  is  much  more  significant.  So,  if  the  decision 
maker  were  not  totally  confident  in  the  weightings  of  Table 
IV  being  correct  as  opposed  to  the  weightings  of  Table  V, 
he  should  analyze  the  problem  more  carefully. 
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I 


Figure  5 
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TABLE  VI 

Combined 


|  Attribute 

Rating 

Alternative  1 

9.23 

2 

6.07 

3 

4.28 

4 

6.58 

5 

3.37 

6 

4.02 

7 

4.  00 

8 

2.43 
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The  final  equivalent  weight  of  each  attribute  as  it 


taken  into  account 

in  Table  IV  is: 

Attribute  a: 

.3 

b: 

.  3 

c: 

( .4)x( .5)  =  .2 

d: 

(.4)x(.5)x(.4)  = 

00 

o 

• 

e : 

( .4)x( .5)x( .6)  = 

.  12 

A  big  advantage  of  this  method  over  the  simple 
assigning  of  all  weights  at  one  step  is  the  ability  to  see 
the  development  of  a  ranking  step  by  step  as  only  some 
attributes  are  traded  off,  so  the  decision  maker  can  better 
understand  the  effect  of  his  chosen  weights.  Also,  at 
critical  steps  the  decision  maker  can  try  several  different 
weight  combinations  to  investigate  how  robust  the  ranking 
is.  Application  of  the  method  is  is  a  fairly 
straightforward  process. 

A  disadvantage  is  that  if  there  are  several  decision 
makers,  consensus  agreement  on  attribute  values  and  relative 
weights  may  be  difficult  to  obtain.  Ideally,  attribute 
ratings  would  be  consistent  so  that  a  rating  of  8  would  be 
as  much  better  than  6  as  6  is  better  than  4. 


T 


T  T 
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A  METHOD  BASED  ON  RELATIVE  POSITION  ESTIMATE 


The  relative  position  estimate  method  is  taken  from 
Reference  2  and  does  not  require  accurate  information 
concerning  attribute  ratings  as  does  the  MOOT  method,  but  it 
may  produce  a  less  refined  ranking.  For  each  attribute,  the 
alternatives  are  judged  relative  to  each  other  in  a  pairwise 
fashion,  and  the  results  are  shown  in  a  preference  matrix. 

If  alternative  i  is  judged  to  be  better  than  or  equivalent 
to  alternative  j,  then  row  i  column  j  of  the  matrix  is  a  1 . 
Otherwise,  if  alternative  i  is  judged  to  be  worse  than 
alternative  j,  then  row  i  column  j  of  the  matrix  is  a  0. 

So,  the  sum  of  elements  across  row  k,  gives  the  number  of 
alternatives  that  are  no  better  than  k,  while  the  sum  of 
elements  down  column  k  gives  the  number  of  alternatives  that 
are  no  worse  than  k.  No  estimates  of  attribute  ratings  need 
be  made,  only  relative  "better  or  worse"  comparisons. 
However,  the  attributes  are  assumed  to  be  independent  of  one 
another . 


The  preference  matrices  used  to  illustrate  the  method 
are  constructed  so  as  to  be  consistent  with  Table  I,  to 
facilitate  comparison  with  the  MOOT  method.  They  are  shown 
in  Tables  VII  to  XI,  with  Table  VII  being  the  preference 
matrix  for  attribute  'a'.  Table  VIII  being  the  preference 
matrix  for  attribute  'b'  and  so  on. 
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TABLE  VII 


1 

2 

3 

4 

5 

6 

7 

8 

1 

1 

1 

1 

1 

1 

1 

1 

1 

2 

0 

1 

1 

0 

0 

0 

1 

1 

3 

0 

1 

1 

0 

0 

0 

1 

1 

4 

0 

1 

1 

1 

1 

1 

1 

1 

5 

0 

1 

1 

0 

1 

1 

1 

1 

6 

0 

1 

1 

0 

1 

1 

1 

1 

7 

0 

0 

0 

0 

0 

0 

1 

0 

8 

0 

0 

0 

0 

0 

0 

1 

1 

Attribute  'a'  (Acquisition  expense) 
Preference  Matrix 

Weight  .3 


TABLE  VIII 


1 

2 

3 

4 

5 

6 

7 

8 

1 

1 

1 

1 

1 

1 

1 

1 

1 

2 

0 

1 

1 

1 

1 

1 

1 

1 

3 

0 

0 

1 

1 

1 

1 

1 

1 

4 

0 

0 

0 

1 

1 

1 

0 

1 

5 

0 

0 

0 

0 

1 

0 

0 

1 

6 

0 

0 

0 

0 

1 

1 

0 

1 

7 

0 

0 

1 

1 

1 

1 

1 

1 

8 

0 

0 

0 

0 

0 

0 

0 

1 

Attribute  'b1  (Operating  expense) 
Preference  Matrix 

Weight  .3 


T 
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TABLE  IX 


Attribute  'c'  (Safety) 
Preference  Matrix 

Weight  .2 


Attribute  'd'  (Effectiveness) 
Preference  Matrix 


Weight  .08 
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TABLE  XI 


1 

2 

3 

4 

5 

6 

7 

8 

1 

1 

0 

1 

1 

1 

1 

1 

0 

2 

1 

1 

1 

1 

1 

1 

1 

1 

3 

0 

0 

1 

1 

1 

1 

1 

0 

4 

0 

0 

0 

1 

0 

0 

0 

0 

5 

0 

0 

0 

1 

1 

1 

0 

0 

6 

0 

0 

0 

1 

0 

1 

0 

0 

7 

0 

0 

1 

1 

1 

1 

1 

0 

8 

1 

1 

1 

1 

1 

1 

1 

1 

Attribute  'e'  (Requirement) 

Preference  Matrix 

Weight  .12 

The  decision  maker  must  choose  weights  for  the  attributes, 
and  using  these  weights  calculations  are  performed  with 
elements  of  the  preference  matrices  to  generate  an  aggregated 
preference  matrix.  Some  simple  calculations  on  entries  of  the 
aggregated  preference  matrix  produces  a  preference  ranking  of 
the  alternatives. 

In  this  example  the  weights  are  chosen  (to  be  consistent 
with  the  resultant  weights  in  the  MOOT  example)  as  .3  for 
attribute  'a',  .3  for  attribute  'b1,  .2  for  attribute  'c',  .08 

for  attribute  'd',  and  .12  for  attribute  'e ' . 

Not  all  of  the  generality  possible  in  the  method  is 
illustrated  here.  Rather,  one  specific  implementation  is  shown 
in  order  to  give  an  idea  of  how  it  works. 
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Let  w(i  >  j)  be  the  sum  of  weights  of  attributes  whose 
preference  matrix  has  row  i  column  j  equal  to  1/  and  row  j 
column  i  equal  to  0  (i.e.  where  alternative  i  is  clearly 
preferred  to  element  j),  and  similarly  let  w(i  < j )  be  the 
sum  of  weights  of  attributes  whose  preference  matrix  has  row 
i  column  j  equal  to  0  and  row  j  column  i  equal  to  1  (i.e. 
alternative  j  is  clearly  preferred  to  alternative  i).  Also, 
let  w(i  =  j)  be  the  sum  of  weights  of  attributes  whose 
preference  matrix  has  both  row  i  column  j  and  row  j  column  i 
equal  to  1  (i.e.  no  clear  preference). 

An  indicator  of  aggregate  preference  that  can  be  used 

is : 

w  ( i  >  j  )  +  w  (  i  =  j) 

A  (  i  >  j  )  = - 

w(i<j)  +  w  (i  =  j) 

One  property  of  this  indicator  is  that 

A ( i>  j  )  =  1/A ( j  <  i  ) 

If  A(i  >j)  =  1  (or  very  close  to  1)  then  it  can  be 
considered  that,  considering  weighted  values  of  attribute 
preferences,  alternatives  i  and  j  are  equally  preferable. 

If  A(i  >  j)  > 1,  then  i  is  considered  preferable  to  j 
(disregarding  all  other  alternatives)  and  if  A(i  > j)  <1, 
then  j  is  considered  preferable  to  i  (disregarding  all  other 
alternatives ) . 
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By  calculating  all  such  A(i  > j)  values,  the 
information  needed  to  construct  an  aggregate  preference 
matrix  is  obtained.  As  an  example,  consider  the  calculation 
of  A ( 2  >  6  ) . 

w ( 2  >  6)  =  .3  +  .2  +  .12  =  .62 

w(2  <  6)  =  .3 

w ( 2  =  6)  =  .08 

.62  +  .08 

and  A  ( 2  >  6)  =  -  =  1.84 

.3  +  .08 

Since  A(2  >6)  >1,  then  2  is  clearly  preferred  to  6  (as  a 
pairwise  comparison)  and  row  2  column  6  of  the  aggregate 
preference  matrix  is  a  1,  and  row  6  column  2  of  the 
aggregate  preference  matrix  is  a  0.  The  other  calculations 
are  not  shown  here,  but  the  resulting  aggregate  preference 
matrix  is  shown  as  Table  XII.  In  working  out  the  values,  if 
A ( i  >  j  )  ranged  from  .9  to  1.11,  then  A ( i  >  j )  was  considered 
(for  practical  purposes)  to  be  1. 
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TABLE  XII 


1 

2 

3 

4 

5 

6 

7 

8 

1 

1 

1 

1 

1 

1 

1 

1 

1 

2 

0 

1 

1 

1 

1 

1 

1 

1 

3 

0 

0 

1 

0 

0 

0 

1 

1 

4 

0 

1 

1 

1 

1 

1 

1 

1 

5 

0 

0 

1 

0 

1 

1 

1 

1 

6 

0 

0 

1 

0 

0 

1 

0 

1 

7 

0 

0 

1 

1 

1 

1 

1 

1 

8 

0 

0 

0 

0 

0 

0 

1 

1 

A  function  '  v'  which  is  an  index  of  relative  position 
can  be  defined  in  terms  of  two  other  functions,  'p'  and 
'q'.  Let  p(k)  be  the  sum  of  the  numbers  in  row  k  of  the 
aggregate  preference  matrix  and  let  q(k)  be  the  sum  of  the 
numbers  in  column  k  of  the  aggregate  preference  matrix. 
p(k)  is  the  number  of  alternatives  which  are  no  better  than 
alternative  k,  while  q(k)  is  the  number  of  alternatives 
which  are  no  worse  than  k.  The  index  v(k)  of  alternative  k 
is  defined  as: 

v  (k  )  =  p  (k  )  -  q  (k  ) 

So,  for  the  aggregate  preference  matrix  of  Table  XII,  the 
indices  are  calculated  as  follows: 

v(l)  =8-1=7 
v  (  2  )  =7-3  =  4 
=  3  -  7  =  -4 


v  (  3 ) 
v  ( 4 ) 


7 


4 


3 
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v  (5 )  =5-5  =  0 

v  (6 )  =  3  -  6  =  -3 

v  (7)  =  6  -  7  =  -1 

v  (8 )  =  2  -  8  =  -6 

The  corresponding  aggregate  preference  ranking  is 
shown  in  Figure  7.  In  comparison  to  the  rankings  produced 
by  the  MOOT  method  (Figures  4,  5,  6)  it  is  most  similar  to 
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Figure  5  even  though  the  attribute  weights  correspond  to 
those  used  for  Figure  4. 

It  is  very  possible  that  in  large  problems  the 
relative  position  estimate  may  generate  an  answer  having 
many  ties.  For  instance,  a  ranking  of  100  alternatives 
might  generate  only  60  distinct  index  values  V(k),  so  that 
40  alternatives  would  have  to  be  tied  in  rank  with  other 
alternatives . 

One  of  the  big  advantages  of  this  method  is  that  it 
can  be  used  when  the  decision  maker  is  unsure  of  placing 
attribute  ratings  on  various  alternatives,  as  long  as  he  is 
willing  to  make  pairwise  comparisons  of  the  alternatives 
with  respect  to  the  various  attributes.  Also,  application 
of  the  method  is  a  simple  and  straightforward  matter. 

A  big  disadvantage  is  that  the  task  of  making 
pairwise  comparisons  would  be  extremely  tedious  if  a  large 
number  of  projects  were  being  compared.  If  100  projects 
were  being  ranked  over  6  attributes,  then  someone  would  have 
to  prepare  6  preference  matrices,  each  having  10,000 
elements.  So,  decisions  on  60,000  pairwise  comparisons 
would  have  to  be  made  and  recorded.  This  workload  could  be 
alleviated  substantially  by  ranking  the  projects  within  5 
groups  of  20,  say,  and  then  taking  the  highest  ranking 
projects  within  each  group  together  for  a  final  overall 
comparison.  However,  the  pairwise  comparison  process  would 
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still  be  somewhat  tedious.  As  with  the  MOOT  method,  the 
difficulty  of  obtaining  consensus  agreement  on  weights  may 
be  a  significant  problem  when  there  are  several  decision 
makers. 

A  FUZZY  SET  METHOD 


The  fuzzy  set  method  is  taken  from  Reference  3  and  is 
a  method  which  can  accomodate  (to  a  degree)  different 
individuals'  opinions  about  attribute  rating  levels  and 
attribute  weights.  Both  weighting  and  rating  levels  can  be 
obtained  by  surveying  members  of  a  committee,  and  combining 
the  opinions  into  one  overall  "fuzzy"  representation.  The 
combined  "fuzzy"  representation  for  each  alternative  is 
compared  with  those  for  other  alternatives,  and  a  priority 
ranking  is  determined.  Again,  attributes  are  assumed  to  be 
independent  of  one  another. 

Let  us  first  consider  the  fuzzy  representation  of  an 
attribute  rating  level.  On  a  two  member  committee,  say  that 
one  member  would  be  100%  satisfied  with  a  rating  level  (for 
some  arbitrary  attribute)  of  6  to  8,  and  at  least  50% 
satisfied  with  any  rating  level  of  4  to  9 ,  and  not  at  all 
satisfied  with  any  other  rating.  A  convenient  way  of 
writing  this  would  be  that  he  has  a  100%  interval  of 
(6,8),  and  a  50%  interval  of  (4,9)  and  his  opinion  can  be 
pictured  as  in  Figure  8.  As  well,  say  the  second  member  has 
a  100%  interval  of  (5,7)  and  a  50%  interval  of  (2,7)  as 
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Figure  8 


100% 

50%  r 

*  *  * 
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Figure  9 


100% 

50% 
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*  *  * 

0 


Figure  10 


25 


******** 

10 


Member  1  -  Attribute  rating 


******** 


10 


Member  2  -  attribute  rating 


Consensus  -  attribute  rating 
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pictured  in  Figure  9.  One  method  of  combining  these  "fuzzy" 
opinions  would  be  to  proportionally  "add"  individual 
representations  together,  and  normalize  the  results  to 
100%.  So,  combining  Figures  8  and  9,  the  resulting  consensus 
representation  would  be  Figure  10,  and  could  be  described  as 
a  100%  interval  of  (6,7),  a  75%  interval  of  (5,7),  a  50% 
interval  of  (4,8),  and  a  25%  interval  of  (2,9). 

Committee  members  opinions  on  attribute  weights  would 
be  combined  in  a  similar  manner  in  order  to  generate 
consensus  weight  representations. 

A  definite  number  of  distinct  levels  is  used  to 
represent  ratings  (instead  of  smooth  curves  or  an  indefinite 
number  of  levels),  and  some  representations  might  have  to  be 
portrayed  as  approximations.  In  practice,  fuzzy 
representations  must  not  have  multiple  "peaks",  or 
computational  problems  will  result. 

As  an  example,  Table  XIII  shows  weight  and  attribute 
rating  representations  (based  loosely  on  Table  I,  to 
facilitate  comparison  with  the  MOOT  method),  at  levels  of 
100%  and  50%,  for  eight  alternatives  having  five 
attributes . 

For  each  alternative  the  consensus  attribute  ratings 
are  compared  with  the  corresponding  weight  representation. 
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TABLE  XIII 


Interval 
Level ( % ) 

Attribute 

a  b 

c 

d 

e 

Weight 

100 

(.2, .4) 

(.2, .4) 

(-1,-3)  (  0 , . 1  ) 

(.1,-2) 

50 

( .1, . 5) 

( .1,. 5) 

( 0, - 4  ) 

(  0  ,  .  2  ) 

(  0 ,  .  3  ) 

Alternative 

1 

100 

(8,10) 

(9,10) 

(9,10) 

(7,9) 

(6,8) 

50 

(7,10) 

(8,10) 

(8,10) 

(6,10) 

(5,9) 

2 

100 

(2,4) 

(8,10) 

(3,5) 

(8,10) 

(7,9) 

50 

(1,5) 

(7,10) 

(2,6) 

(7,10) 

(6,10) 

3 

100 

(2,4) 

(7,9) 

(0,1) 

(1,3) 

(4,6) 

50 

(1,5) 

(6,10) 

(0,2) 

(0,4) 

(3,7) 

4 

100 

(7,9) 

(6,8) 

(7,9) 

(3,5) 

(0,1) 

50 

(6,10) 

(5,9) 

(6,10) 

(2,6) 

(0,2) 

5 

100 

(4,6) 

(0,2) 

(2,4) 

(9,10) 

(1,3) 

50 

(3,7) 

(0,3) 

(1,5) 

(8,10) 

(0,4) 

6 

100 

(4,6) 

(3,5) 

(1,3) 

(8,10) 

(0,2) 

50 

(3,7) 

(2,6) 

(0,4) 

(7,10) 

(0,3) 

7 

100 

(0,1) 

(7,9) 

(2,4) 

(4,6) 

(4,6) 

50 

(0,2) 

(6,10) 

(1,5) 

(3,7) 

(3,7) 

8 

100 

(1,3) 

(0,1) 

(1,3) 

(8,10) 

(7,9) 

50 

(0,4) 

(0,2) 

(0,4) 

(7,10) 

(6,10) 

FUZZY  REPRESENTATIONS  OF  PARAMETER  VALUES 


Note:  Attribute  weight  intervals  are  between  0  and  1,  while  attribute 

rating  intervals  are  between  0  and  10. 
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and  an  overall  rating  representation  is  created.  At  the 
100%  level,  the  maximum  and  minimum  endpoints  of  the 
weighting  representations  are  considered  in  combination  with 
the  maximum  and  minimum  endpoints  of  the  corresponding 
attribute  representations,  in  order  to  produce  upper  and 
lower  end-points  of  an  overall  rating  at  the  100%  level. 
Then,  a  similiar  process  is  carried  out  to  produce  upper  and 
lower  endpoints  of  the  overall  rating  at  the  50%  level. 
Details  of  the  process  are  described  in  Annex  A.  The 
overall  ratings  are  listed  in  Table  XIV,  and  pictured  in 
Figure  11.  A  final  priority  ranking  is  obtained  by 
computing  the  centroid  of  each  overall  rating  representation 
(Table  XIV) ,  and  ordering  the  alternatives  from  most 
desirable  to  least  desirable  by  decreasing  value  of 
centroids  (Figure  12).  This  ranking  is  almost  identical  to 
the  one  produced  by  the  MOOT  method  (Figure  4). 


-r 
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TABLE  XIV 


Interval 

Level 

Overall  Rating 

Centroid 

Alternative  1 

100 

(7. 8, 9. 8) 

8.3 

50 

(6,10) 

2 

100 

(4,8.1) 

5.9 

50 

(2,9.5) 

3 

100 

(2. 5, 6. 4) 

4.8 

50 

(0.9, 9. 2) 

4 

100 

(4.5,8) 

6.1 

50 

(2. 1, 9. 9  ) 

5 

100 

(1.3,5) 

3.7 

50 

(.3,7.5) 

6 

100 

(1.9, 5. 4) 

4.2 

50 

(.6,8.3) 

7 

100 

(2.4,6) 

4.6 

1 

50 

(1,8.7) 

i 

8 

100 

(1. 2,4. 9) 

3.7 

! 

50 

(0,8) 

native 


Alternative 

3 


native 


mg 
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Priority  Ranking 
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Perhaps  the  biggest  advantage  of  using  this  method  is 
that  it  can  avoid  problems  associated  with  obtaining 
consensus  agreement  on  attribute  ratings  and  relative 
weights. 

A  disadvantage  with  large  problems  could  be  the 
number  of  intervals  that  must  be  elicited  and  recorded, 
before  computation  can  proceed.  On  a  problem  with  100 
alternatives,  each  having  10  attributes  to  be  rated  by  5 
decision  makers,  with  rating  intervals  specified  at  4  levels 
(25%,  50%,  75%,  100%),  a  total  of  20,000  rating  intervals 
would  have  to  be  elicited  (4,000  from  each  decision  maker). 
Then,  they  would  have  to  be  typed  as  input  data  to  a 
computer  program  and  checked  for  errors.  The  effort 
required  to  generate  this  input  data  might  in  such  a  case 
be  considered  prohibitive.  A  similar  problem  with  rating 
intervals  specified  at  only  two  levels  by  two  (instead  of 
five)  decision  makers,  would  be  much  more  manageable. 

ASSESSMENT  OF  AN  ADDITIVE  VALUE  FUNCTION  USING  CONJOINT 
SCALING 

Consider  again  the  example  shown  in  Table  1.  For 
some  arbitrary  alternative,  let  the  values  of  attributes 
'a',  'b'  ...,  'e'  be  represented  by  Xa,  X5,  ...,  Xe.  Then, 

under  certain  conditions  a  value  function  '  v'  which 
signifies  the  desirability  of  an  alternative  can  be 
constructed  in  terms  of  separate  functions  '  va'  through 
' ve ' ,  such  that 

v(Xa/  xb'  ...,  Xe)  =  Va(Xa)  +  vb(xb)  +  ...  +  Ve(Xe) 


T 


T 
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This  form  of  the  function,  which  is  represented  by  the 
summation  of  functions  of  individual  parameters,  is  referred 
to  as  additive.  So  once  the  functions  1 va ' ,  through  ' ve ' 
are  determined,  the  relative  value  of  any  alternative  can 
quite  easily  be  calculated,  and  these  values  can  be  used  to 
construct  a  priority  ranking.  The  difficult  part  is  the 
effort  that  must  expended  in  order  to  obtain,  assimilate, 
and  verify  the  information  needed  to  form  '  va'.  through 
ve  • 


Some  conditions  must  be  satisfied  before  constructing 
an  additive  value  function.  First,  any  pair  of  attributes 
must  be  "preferentially  independent"  of  the  others.  Second, 
relative  trade-offs  over  the  entire  possible  ranges  of  the 
five  attributes  must  be  determined.  These  conditions  are 
now  discussed  in  further  detail. 

Let  the  attribute  ratings  of  an  alternative  be 
represented  by  an  ordered  quintuple  ('a'  is  the  first 
element,  'b'  is  the  second,  and  so  on).  Say  that  one 
attribute  having  ratings  (5,  7,  X,  Y,  Z)  is  preferred  over 
another  having  ratings  (3,  9,  X,  Y,  Z)  for  some  specific  X, 
Y,  Z.  If  the  first  pair  of  attributes  is  preferentially 
independent  of  the  rest,  then  for  any  X,  Y,  Z,  an 
alternative  having  ratings  (5,  7,  X,  Y,  Z)  is  preferred  over 
one  with  ratings  (3,  9,  X,  Y,  Z).  An  explanation  of  how  the 
preferential  independence  property  is  intrinsically  related 
to  an  additive  value  function  is  given  in  Reference  4. 
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The  specification  of  trades-offs  over  the  ranges  of 
the  attributes  involves  extensive  and  detailed  examination 
of  attribute  interrelationships.  One  approach  in  doing  this 
is  to  determine  the  relative  value  of  one  attribute  at 
certain  ratings,  with  respect  to  another  attribute  at 
specific  ratings.  For  instance,  if  one  alternative  had  an 
'a'  value  of  5,  and  a  'b'  value  of  7  and  a  second 
alternative  had  an  'a'  value  of  4,  the  decision  maker  would 
have  to  determine  which  value  for  'b'  would,  with  all  other 
attributes  the  same,  make  the  second  alternative  just  as 
desirable  as  the  first.  Another  way  of  looking  at  this 
would  be  to  say,  "If  I  devalued  the  'a'  attribute  from  5  to 
4,  how  much  would  I  have  to  increase  'b1  from  7  in  order  to 
compensate?"  Such  trade-offs  must  be  done  for  all  pairs  of 
attributes.  Then,  checks  for  consistency  must  be  made.  One 
form  of  consistency  check  is  to  ensure  that  for  any  pair  of 
attributes  no  trade-off  specified  at  certain  ratings  is  in 
contradiction  with  any  other  specified  trade-offs.  Another 
form  of  consistency  check  is  to  ensure  that  larger  groups  of 
attributes  are  well  determined.  For  instance,  the 
trade-offs  for  attributes  'a'  and  'b1,  as  well  as  the 
trade-offs  for  'b*  and  'c',  should  not  contain  any 
contradictions  with  respect  to  the  trade-offs  between  'a1 
and  '  c  ' . 


In  order  for  this  method  to  be  useful,  the  parameter 
values  specified  for  each  alternative  should  be  known  to  be 
accurate,  and  the  decision  maker  must  have  faith  in  the 
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trade-offs  that  are  determined.  Since  the  essence  of  this 
method  is  in  the  compiling  of  trade-offs,  it  is  not  so 
readily  illustrated  as  the  other  methods,  and  the  interested 
reader  should  consult  Reference  4.  This  source  also 
illustrates  the  construction  of  the  additive  value  function 
after  completing  the  trade-off  elicitation  process. 

An  advantage  of  conjoint  scaling  is  that  it  will, 
with  accurate  trade-off  and  rating  values,  provide  an 
accurate  ranking. 

A  disadvantage  is  that  the  extensive  trade-off 
determination  and  checking  of  responses  for  consistency 
would  be  time  consuming,  and  would  probably  require  a  very 
talented  analyst.  If  several  decision  makers  were  to  be 
involved,  there  would  likely  be  even  more  difficulty  in 
obtaining  consensus  agreement  than  there  would  be  with  the 
other  ranking  methods. 

SUMMARY  OF  METHODS 


Each  one  of  the  four  methods  described  in  this  note 
may,  under  certain  conditions,  be  more  desirable  than  any  of 
the  others. 

If  attribute  levels  for  each  alternative  can  be 


specified  with  accuracy,  and  if  trade-offs  can  be 
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ascertained  accurately,  then  the  construction  of  an  additive 
value  function  using  conjoint  scaling  would  give  the  most 
precise  priority  ranking. 

On  the  other  hand,  if  attribute  levels  are  reasonably 
exact,  and  the  decision  maker  is  only  moderately  confident 
in  assigning  weights  to  attributes,  the  MOOT  method  is  then 
preferable.  It  more  readily  demonstrates  the  ramifications 
of  assigned  weights,  and  allows  some  sensitivity  analysis. 

If  the  decision  maker  is  only  moderately  confident  in 
both  attribute  levels  and  attribute  weights,  the  fuzzy  set 
method  described  here  would  be  most  appropriate. 

However,  if  attribute  levels  cannot  be  confidently 
assigned,  then  the  relative  position  estimate  method  can  be 
of  use  provided  the  appropriate  pairwise  comparisons  can  be 
made,  and  reasonable  weights  can  be  assigned  to  the 
attributes.  For  the  relative  position  estimate,  and 
possibly  for  the  MOOT  method,  the  Analytic  Hierarchy  Process 
described  in  Reference  5  provides  a  procedure  which  is  well 
worth  considering  for  determining  attribute  trade-off 
weights . 


Accuracy  requirements  and  implementation  effort  are 
summarized  for  the  four  methods  in  Tables  XV  and  XVI.  For 
repeated  applications  of  any  of  the  methods,  computer 
assistance  would  be  desirable  for  sizable  rankings,  while  it 
may  be  feasible  to  do  one-time  applications  of  some  methods 


with  a  calculator. 


TABLE  XV  -  Accuracy  Requirement 


MOOT 


Relative 

Postion 

Estimate 


Fuzzy 

Set 

Method 


Attribute 

Rating  Levels 

Attribute  Weight  or  Tradeoff 

Accurate 

Specification 

Moderately 

Accurate 

Vague 

Accurate 

Specification 

Moderately 

Accurate 

Vague 

Conjoint 

Scaling 
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ANNEX  A 

CONSTRUCTION  OF  OVERALL  RATING  REPRESENTATIONS 
IN  THE  FUZZY  SET  METHOD 


The  construction  of  an  overall  rating  representation 
is  explained  here  by  means  of  an  example.  The  calculation 
of  the  overall  100%  level  upper  and  lower  endpoints  for 
Alternative  2  (taken  from  the  example  in  the  main  text) 
is  shown.  Calculation  of  the  50%  level  endpoints  would  be 
done  in  similar  fashion. 

The  100%  level  intervals  for  attribute  weights 
and  Alternative  2  ratings  are  reproduced  below: 

TABLE  I 


Interval 

Level 

a 

Attribute 
b  c 

d 

e 

Weight 

100% 

(.2, .4) 

(.2,  .4)  ( . 1 , . 3 ) 

(0, .1) 

( .1, .2) 

Alternative  2 

100% 

(2,4) 

(8,10)  (3,5) 

(8,10) 

(7,9) 

The  upper  endpoint  of  the  overall  rating  at  the  100% 
level  is  an  expression  of  the  form 


r  = 


. Z,  w .r  . 
0=1  3  3 

__ 

£  w . 

j=l  D 


where  r^  is  the  upper  endpoint  of  the  jth  attribute  rating 
at  the  100%  level,  and  w^  is  either  the  lower  or  upper 
endpoint  of  the  jth  weight  at  the  100%  level.  The  w.. 
values  are  chosen  so  as  to  maximize  r,  and  are  determined 
with  a  set  of  computations  which  are  explained  presently. 


A-l 


.  . ./A- 2 
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A  series  of  five  calculations  (equal  to  the  number 
of  attributes)  is  performed.  In  the  first  one,  r 
is  calculated  with  being  the  upper  endpoint  for  k 
corresponding  to  a  highest  rating  value  r^,  and  all  other 
Wj  values  (j  ^  k)  as  lower  endpoints.  In  the  second 
calculation,  r  is  worked  out  with  w^  and  w^  as  upper 
endpoints  for  those  values  of  k  and  1  which  correspond 
to  the  two  highest  rating  values  r^  and  r^,  and  all 
other  Wj  values  (j  /  k  or  1)  are  lower  endpoints.  This 
pattern  continues,  with  successive  w^  values  for  the  third- 
highest,  fourth-highest ,  etc.  r^  values  being  switched 
from  the  lower  to  the  higher  endpoint.  It  can  be  shown 
that  the  maximum  r  value  calculated  amongst  such 
successive  trials  is  indeed  the  maximum  r  value  that  can 
be  generated  by  choosing  any  combination  of  w^  and  r^ 
values  which  are  within  the  specified  intervals.  The 
calculation  results  are  shown  in  Table  II. 


A- 2 


.  .  .  /A- 3 
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RIAL 


TABLE  II 

MAXIMUM  r  CALCULATIONS 


Maximum  r  value  is  8.1 

TABLE  III 


MINIMUM  r  CALCULATIONS 


9  7.8 

9  8.0 

9  8.1 


.4 

2 

.2 

8 

.1 

3  ( 

)  8 

.1 

7 

4.3 

.4 

2 

.2 

8 

.3 

3  ( 

)  8 

.1 

7 

4.0 

.4 

2 

.2 

8 

.3 

3  C 

)  8 

.2 

7 

4.3 

.4 

2 

.4 

8 

.3 

3  ( 

)  8 

.2 

7 

00 

• 

.4 

2 

.4 

8 

.3 

3  .] 

L  8 

.2 

7 

5.1 

Minimum  r  value  is  4.0 
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In  calculating  the  lower  endpoint  of  the  overall 
rating  at  the  100%  level,  the  same  expression  for  r  is 
used,  but  the  r^  values  are  the  lower  endpoints  of  attribute 
ratings,  and  w^  values  are  chosen  so  as  to  minimize  r. 

Again,  a  number  of  calculations  equal  to  the  number 

of  attributes  are  performed.  In  the  first  one,  r  is 

calculated  with  wfc  being  the  upper  endpoint  for  k  corresponding 

to  a  lowest  rating  value  r,  ,  and  all  other  w.  values  (j  ^  k) 

K  D 

as  lower  endpoints.  In  the  second  calculation,  r  is  worked 
out  with  wk  and  w^  as  upper  endpoints  for  those  values  of 
k  and  1  which  correspond  to  the  two  lowest  rating  values 
rk  and  r^r  and  all  other  w..  values  (j  /  k  or  1)  are  lower 
endpoints.  This  pattern  continues,  with  successive  w^  values 
for  third-lowest ,  fourth- lowest,  etc.  r^  values  being 
switched  from  the  lower  to  the  higher  endpoint.  It  can  be 
shown  that  the  minimum  r  value  calculated  amongst  such 
successive  trials  is  indeed  the  minimum  r  value  that  can 
be  generated  by  choosing  any  combination  of  w^  and  r^  values 
within  the  specified  intervals.  The  calculation  results 
are  shown  in  Table  III. 

So,  the  overall  rating  of  Alternative  2  at  the  100% 
level  can  be  represented  by  the  interval  (4.0,  8.1) . 

Interval  calculations  at  the  50%  level  are  done  in  the 

same  way  as  the  preceding  presentation,  using  the  appropriate 

weight  and  rating  upper  and  lower  endpoints. 


A- 4 
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